Agents with a Moral Dimension (Doctoral Consortium)

As argued by [9], moral decision making entails considering alternatives and assessing the pros and cons of their possible consequences for self and others. From the area of affective neuroscience the concept of moral emotions has been introduced [9] and neurobiological findings [7] show that moral emotions are used to judge the adequacy of actions and are central to moral behavior, decision making and learning. My aim is to build a computational model for moral emotions in order to enable intelligent agents [2] to understand moral consequences of actions through moral emotions. The agent is able to compare alternative scenarios and to decide what course of actions and goals to pursue in order to show a morally driven behavior. Moral emotions are useful when the agent is engaged in a social interaction with a user or other agents, because (i) moral emotions may lead the agent towards the compliance with (shared) moral values (ii) the agent is equipped with moral emotions which make her potentially emphatic to others.